Sparse recovery by reduced variance stochastic approximation

Abstract In this paper, we discuss application of iterative Stochastic Optimization routines to the problem sparse signal recovery from noisy observation. Using Mirror Descent algorithm as a building block, develop multistage procedure for solutions under assumption smoothness and quadratic minoration on expected objective. An interesting feature proposed is linear convergence approximate solution during preliminary phase routine when component stochastic error in gradient observation, which due bad initial approximation optimal solution, larger than ‘ideal’ asymptotic owing observation noise ‘at solution’. We also show how one can straightforwardly enhance reliability corresponding using Median-of-Means-like techniques. illustrate performance algorithms classical problems low-rank signals generalized regression framework. show, rather weak regressor distributions, they lead parameter estimates obey (up factors are logarithmic dimension confidence level) best known accuracy bounds.